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Abstract — We consider a set of flows passing tlirougli a set 
of servers. The injection rate into each flow is governed by a 
flow control that increases the injection rate when all the servers 
on the flow's path are empty and decreases the injection rate 
when some server is congested. We show that if each server's 
congestion is governed by the arriving traffic at the server then 
the system can oscillate. This is in contrast to previous work on 
flow control where congestion was modeled as a function of the 
flow injection rates and the system was shown to converge to a 
steady state that maximizes an overall network utility. 

Categories and subject descriptors: F.2.2 [Nonnumerical 
Algorithms and Problems]: Sequencing and scheduling. 
General Terms: algorithms. 
Keywords: routing, scheduling, stability. 

I. Introduction 

In this paper we study how TCP-like mechanisms for flow 
control operate in a network of queues with feedback. We 
consider a set of flow|3 passing through a network of finite 
capacity servers. Let Xi{t) be the injection rate of data into 
flow i. A series of papers, (e.g. [1], [2], [3], [4], [5], [6], [7], 
[8], [9], [10]) has shown that many variants of TCP can be 
viewed as solving optimization problems on the variables Xi 
subject to the constraint that none of the server capacities are 
overloaded. 

To give a concrete example, consider the following model 
of TCP-like additive-increase-multiplicative-decrease (AIMD) 
flow control considered by Kelly et al. in [2]. Let be the 
service rate of server Q. Let S{Q) be the set of flows that 
pass through server Q. In [2], the injection rate into flow i is 
controlled by 
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In the above equations k and Wi are parameters, Xi{t) repre- 
sents ^Xi{t), 'J2iGS(Q) •^j(^) represents the current congestion 
at server Q and pqiy) is a price charged by server Q when 
the congestion is y. An example given in [2] is 

PQiy) = (y - t^Q + s)^ / 
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for some suitable parameter e. Kelly et al. show in [2] that 
all trajectories of the above system converge to a stable point 
that maximizes the utility function, 

U{x)^ywi log Xi-y / pq {y)dy. 

Q ^0 

For sufficiently small values of e this is approximately equal 
to solving the system, 

subject to: 

J2^eS{Q) ^ MQ V£. 

Since [2] appeared there has been much other work showing 
that many other variants of TCP solve an appropriately defined 
utility maximization problem. However, a common feature of 
this work is that the behavior of servers in the network is 
directly affected by the injection rates of the flows that pass 
through the server In other words, the congestion at server 
Q is modeled as a function of X]ies(Q) We refer to 

such models as injection-rate based models. These models 
are often motivated by networks where congestion is noticed 
and signaled to the flow endpoints before the link is saturated. 
In this case queues never build up. 

In most deployed networks however, congestion is only 
noticed by TCP when a server is saturated and the associated 
queue does build up. In this case the queue dynamics mean 
that the flow arrival rates at internal servers in the network 
may be different than the external flow rate. In order to 
capture such situations, in this paper the congestion will be 
a function of X]ieS(Q) °'(j,Q)(^)' where a(i,Q) is the arrival 
rate of flow i at server Q at time t. We refer to a model that 
models congestion in this way as an arrival-rate based model. 
As already mentioned, the queueing behavior in a network 
can mean that q) (t) behaves dramatically differently than 
Xi{t). Indeed, a large body of work (e.g.[ll], [12], [13], [14], 
[15]) has looked at instability phenomena in networks. In 
these phenomena, the queueing behavior at the servers causes 
an unbounded amount of data to build up in the network, 
while at the same time the injection rates into the network 



do not saturate any link, i.e. ^ 



i{t) < pLQ for afl 

t and Q. This means that even though server capacities are 
never violated, queue occupancies and end-to-end delays grow 
without bound. The reason that these instabilities occur is that 
the arriving traffic creates oscillations in the network. Each 
queue alternates between periods when it is empty and periods 
during which a lot of data arrives, during which the queue 
grows large. These oscillations grow longer with time and so 
the queue sizes grow larger with time. 
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In this paper we examine whether such oscillations can 
occur when the injection rates are governed by a flow control 
such as ([T]i-(|2]i, in the case when the congestion at a link is 

governed by Eies{Q) Hi.Q)i*) rather than Ei:i6S(Q) ^»(*)- 
In particular we consider a network of servers, most of which 
serve data according to First-in-First-Out (FIFO). The rate at 
which data arrives at internal nodes of the network is governed 
by the dynamics of the queues. When data leaves the network 
the amount of delay that the data has experienced within the 
network affects the new injection rates into the network, in a 
manner similar to ([T]i-(|2|i. Our main result is that there exists an 
initial state of the network from which the system has periodic 
oscillations. This in turn means that the flow control never 
converges to an optimal set of injection rates. Moreover, the 
oscillations in our example are very wide: all injection, arrival 
and departure rates, and all queue heights, oscillate between 
and their respective maximal values. 

We stress that the oscillations in our example are solely 
due to the queueing dynamics. This provides a contrast to 
the oscillating examples of Choe and Low [7] and Liu et 
al. [8] in which oscillations occurred due to the introduction 
of propagation delays into an injection-rate based model. We 
discuss this distinction in more detail in Section iLDl 



A. The model: network dynamics 

We consider a set of servers and a set of flows. Each flow 
consists of a path through the network. For ease of analysis 
we model data as a continuous fluid. We begin by describing 
how we model the network dynamics. We then describe how 
we model the flow control. 

Let us consider a given server Q. We will use the following 
notation. We let /i be the service rate of Q, and S be the set of 
flows that pass through Q. For a time t, we define h{t) to be 
the height of server Q, which is the amount of fluid queued at 
server Q normalized by the service rate. We also let ai{t) and 
di{t) be the arrival and departure rates, respectively, of fluid 
on flow i at server Q. 

Most of the servers in our oscillating examples will be 
FIFO servers. A FIFO server Q is governed by the following 
equations (see e.g. [16]). 



d^{t + h{t)) 
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If h{t) > 0, 



If h{t) = 0, 



h{t) 
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That is, any fluid that arrives at server Q at time t will depart 
at time t + h{t). The departure rate of this fluid will equal the 
arrival rate divided by a quantity that is proportional to the 
aggregate arrival rate. 

When fluid leaves server Q it goes to the next server on its 
path, call it Q'. The arrival rate at Q' is determined by the 



departure rate from server Q as well as the propagation delay 
between the two servers, call it r. In particular we have 

In our construction all propagation delays will be zero. 

In addition to the FIFO servers, a small number of servers in 
the network divide the flows into two classes and give priority 
to the first class over the second class; we call them two- 
priority FIFO servers. For these servers the flows from the first 
class behave as if they are being served by a FIFO queue but 
the only arrivals come from the first class flows. The second 
class flows are served according to FIFO among themselves 
but the service rate available to them is limited to the amount 
of capacity that is not used by the first class flows. 

In Section [Vl] we conjecture that these priorities are not 
necessary and that our oscillating example can be modified 
so all the queues are FIFO. The difficulty in proving this 
conjecture lies in arrival rates that are negligibly small but 
cumbersome to analyze. 

B. The model: parameterized flow control 

We now describe our flow control mechanism. Consider 
some session 5 at a given time t. Say it is blocked if at least 
one server Q on its path stores either a positive amount of 
session S fluid, or a positive amount of some other session's 
fluid whose priority at Q is higher than that of S. Say session S 
is happy if any server Q on its path does not store any session 
S fluid and, moreover, the bandwidth available to S* at Q given 
this server's priorities is strictly larger than the injection rate 
of S into Q. Say session S is stable if it is neither blocked nor 
happy, i.e. if it is bottlenecked at the current injection rate. We 
assume that the injection rate of S should increase when S is 
happy, decrease when S is blocked, and stay constant when 
S is stable. 

For the purposes of this paper we will assume that for a 
given session the flow control mechanism is given by two 
functions, f{e,t) and g{t), which control flow decrease and 
flow increase, respectively. Specifically: 

• /(e, t) is the injection rate at time t assuming that at time 
the session becomes blocked and stays blocked till time 
t, and at time the injection rate is e. 

• g{t) is the injection rate at time t assuming that at time 
the injection rate is 0, and between times and t the 
session is happy. 

We will allow an arbitrary flow increase function g{t) as 
long as it is monotone and increases to infinity. We consider 
two main examples of flow decrease: additive decrease: 



f5{e,t) 



5t, 



and multiplicative decrease 



fsis,t) 



ee 



-St 



t<to 

otherwise 



(6) 



(7) 



where a G (0, 1) is a small constant and to is the time such that 
g-i5to — Q, Wg Q instead of the "simple" multiplicative 
decrease fs{e,t) = ee"** because for technical reasons we 
need the injection rate to drop to in some finite time. 
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In both examples (5 > is the damping parameter which 
controls the speed at which the flow rate decreases. We will 
assume that we are given a pair of functions (/, g) which all 
servers must use, but we are allowed to tune 5. To make the 
model slightly more general, we will assume that the flow 
increase function g{t) is also parameterized by this 5. 

With the above types of flow control we can model 
both additive-increase-additive-decrease (AIAD) and additive- 
increase-multiplicative-decrease (AIMD) types of flow control. 
We recall that AIMD schemes have been widely studied 
in connection with providing effective flow control in the 
internet. (See e.g. [17], [18].) 

To generalize examples (|6]l and (|7]i, and to treat them in 
a unified model, we define the flow control mechanism as 
a triple {fs,gs, <5'del), where 5del C M is the set of possible 
values for the damping parameter S, and the functions {fs, gs) 
satisfy the following set of axioms: 

(Al) increase to oo: gs{t) increases to infinity for any 5. 
(A2) decrease to 0: for any fixed 5 and e, function fs{e,t) 

becomes at some positive time, and then stays 0. 
(A3) monotonicity: function fs{e,t) is increasing in variable 

e and decreasing in parameter S and in variable t. 
(A4) damping: for any fixed rates < e' < e and any fixed 

time t, there exists some 6 such that fs{e,t) > e', and 

there exists some other 6 such that fs{e,t) < e'. 
(A5) continuity: for any fixed 6 the function fs{e,t) is a 

continuous function from to K. 

Axioms (A[T}A|3]i are very natural. Axiom (A|4]l expresses 
why it is useful to have a damping parameter. Finally, Ax- 
iom (AID captures the intuition that in the fluid-based model 
everything ought to be continuous. All five axioms easily sat- 
isfied by additive decrease © and multiplicative decrease (|7]i. 

Axiom (AdJ is the only axiom on flow increase; note that 
it says nothing about the damping parameter. We will tune 5 
solely to obtain the desired flow decrease. In particular, for 
flow increase we will tolerate an arbitrary dependence on 5. 

Note that the function fs{e,t) does not need to be contin- 
uous in 6, and the set 5del of possible values of 6 does not 
even need to be a continuous real interval. For example, both 
for additive decrease (|6]l and for multiplicative decrease ^ 
this set can be discrete, e.g. Sbel = N U {I/71 : n <E N}. 

C. The main result and extensions 

A network of servers and sessions is a network of servers 
connected by server-to-server links, together with a set of 
sessions; here each session is specified by a source-sink pair, 
a flow path, and a flow control mechanism. At a given time 
the state of such network includes: 

• injection rates into each session, 

• height and composition of each queue 

Here the composition of a given queue Q includes, for each 
height h and each session S passing through Q, the density 
of session S fluid at height h. (This density is defined to be 
the rate at which session S fluid leaves queue Q after time h.) 
We say that a network oscillates if starting from some state it 
eventually reaches this state again. 



We say that at a given time the injection rates are feasible 
if they do not overload any server: for each server Q, the sum 
of the current injection rates of all sessions that go through Q 
is no bigger than the service rate of Q. 

Our main result is that given any parameterized flow con- 
trol mechanism, we can create a network that exhibits wide 
oscillations during which the injection rates stay feasible. We 
state this result as follows: 

Theorem I.l Suppose we are given an arbitrary flow control 
mechanism that satisfies properties f/Q/Q, and we are al- 
lowed to choose the damping parameter separately for each 
session. Suppose all servers must be either FIFO or two- 
priority FIFO, and we are allowed to choose the service rate 
separately for each server 

Then there exists a network of servers and sessions that 
OSCILLATES SO that the injection rates are feasible at all 
times. The damping parameters take only three distinct values. 

The oscillations are very wide, in the sense that all injection, 
arrival and departure rates, and all queue heights, oscillate 
between and their respective maximal values. 

It is interesting to ask if we can fine-tune the above result 
so that all sessions are really running the same flow control 
mechanism, i.e. have the same damping parameter. We can do 
it for the case of additive decrease (|6]l and more generally for 
all flow decrease functions of the form 

f5{e,t)=e~S-e°' ■ t^, where a < 1 and /? > 0. (8) 

Note that any such function satisfies axioms (A[T}A|5]i. In 
particular, by differentiating we can see that it is increasing in 
£ whenever it is non-negative. 

Theorem 1.2 Suppose in Theorem 17.71 the flow decrease func- 
tion satisfies dSj. Then we can choose the same damping 
parameter for all sessions. Moreover, there exists 5* G 5del 
with the following property: for any 5 6 iSdel such that 5 < 5* 
we can choose the damping parameter to be 5. 

In fact, our result applies to a somewhat more general 
family of flow decrease functions, see Section [V] for further 
discussion. Note that we can take any given 5 as the common 
damping parameter, as long as it is sufficiently small. 

D. Remarks 

We stress that in our example it is the dynamics of the 
queues that cause the oscillations. This provides a contrast 
with the work of Choe and Low [7] and Liu et al. [8] that 
showed that injection-rate based models of TCP Vegas and 
a variant called Stabilized Vegas can both be unstable when 
feedback delays are present. These papers model TCP as 
a set of differential equations with feedback and show that 
the feedback delays can cause the equations to oscillate. In 
particular, for each server Q the injection rates of flows in 
S{Q) oscillate between values that overload Q and values 
that underload Q. In contrast, in our oscillating examples the 
injection rates Xi{t) are such that '}2iieS(Q) ^ l^Q always 
holds and so the injection rates do not a priori overload any 
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server. The oscillations arise because the queueing dynamics 
mean that temporary congestion is continually being created 
in the network. 

Another feature of our model is that when congestion occurs 
the injection rate into a session decreases continuously. Some 
previous work has looked at a contrasting model of TCP in 
which the window size (and hence the injection rate) of a ses- 
sion is instantaneously decreased by half whenever congestion 
occurs. Baccelli et al.[19] showed that these "jumps" can lead 
to oscillations when a system of TCP sessions interacts with a 
RED or drop-tail queue. Our example shows that even if these 
jumps are eliminated using a continuous decrease then the flow 
control can still oscillate when interacting with a network of 
servers. 

One paper that considers a similar problem to ours is 
Ajmone Marsan et al. [20]. They present an example con- 
sisting of a network of finite-buffer servers in which the 
scheduling discipline is either strict-priority or Generalized 
Processor Sharing. The example is constructed in such a 
way that if the source rates are non-adaptive then the queue 
sizes will build up in an oscillating manner. The paper [20] 
demonstrates via simulation that if we instead use adaptive 
additive-increase-multiplicative-decrease sources then we still 
get queue buildups that lead to packet losses and oscillating 
behavior There are a number of differences between the 
model of [20] and our model however The main difference 
is that [20] uses finite buffers and the sources only adapt to 
packet losses. In our model we consider unbounded buffers 
but the sources respond directly to congestion. Our results 
therefore demonstrate that as long as sources respond to 
congestion on a link, we can still get oscillating behavior and 
hence suboptimal utilization, even if no packet losses occur. 
Another difference between the two models is that [20] uses 
networks of strict-priority or GPS servers whereas our servers 
are mostly FIFO. 

Finally we remark that one of the main reasons that we 
are able to create oscillations is that the queueing disciplines 
at the servers are oblivious to the state of the flow control. 
In contrast, there exist schemes in which the flow control 
protocols and the queueing protocols work in combination and 
are able to ensure convergence and prevent oscillations. See 
for example the Greedy primal-dual algorithm of Stolyar [21]. 

E. Preliminaries 

Without loss of generality we may assume that we can 
choose the maximal injection rate for a given session. Indeed, 
for any target maximal rate e we can attach a new server in the 
very beginning of the session path, with a service rate e. For 
simplicity, in the following sections each session will have 
two flow control parameters (e, S), where e is the maximal 
injection rate and S is the damping parameter 

In practical settings it makes no sense to have sessions with 
non-simple flow paths (i.e. flow paths that go through the same 
server more than once). However, non-simple flow paths are 
often useful theoretically since they can make a construction 
more compact and/or clear. It turns out that one can use non- 
simple flow paths without loss of generality, since any network 



with non-simple flow paths can be converted to an equivalent 
network where all flow paths are simple, e.g. see [22]. The 
basic idea is that if Z is the maximum number of times that a 
session visits a server then we create Z copies of the network. 
Whenever a session is about to visit a server for second time it 
simply moves to a new copy of the network. More precisely, 
we organize the copies into a ring: we number the copies 
from to Z — 1, and the next visit from copy i goes into copy 
{i + 1) (mod Z). In this way we can create an example in 
which each session visits a server at most once. However, in 
order to avoid this extra complexity, we will use non-simple 
flow paths without any further notice. 

F. Organization of the paper 

In Section [U] we overview our construction. In Section |lll] 
we describe the main building block in our construction, which 
we call the basic gadget. Then in Section|lV]we proceed to the 
full construction and the proof of its performance. In SectionlVl 
we discuss the extension where we choose the same damping 
parameters for all session. In Section [Vl] we conjecture an 
extension where all servers are FIFO. Finally, in Section IVIII 
we conclude and state some open questions, 

II. Overview of Oscillating Example 

The complete description of our oscillating example is 
somewhat complex and so we begin with a brief overview. 
Our construction is based on rows of gadgets. Each row of 
gadgets has a single horizontal session H that passes through 
all the servers in the row multiple times. Each gadget in a row 
consists of four servers Qi, Q2, Q3 and Q4. (See Figure [U. 
The server Q4 in one gadget is identified with the server Qi 
in the next gadget. The main aim of the gadget is to transfer 
session H fluid from server Qi to server Q4. In order that 
this can occur session H loops multiple times through all of 
servers Q1-Q4 and then loops multiple times through server 
Q4 only. We also have a session S that goes through server 
Q4 only. 

Initially, server Qi contains session H fluid and server Q4 
is empty. This means that session S is not blocked and so 
it is injecting at its maximum rate. As session H fluid loops 
through server Q4 we get a buildup in server Q4. This causes 
the injection rate of session S to decrease. Eventually we reach 
a state in which all of the session H fluid is now in Q4, the 
injection rate of session S is zero, and there is no session S 
fluid in server Q4. 

Since server Q4 in the current gadget is identified with 
server Qi in the next gadget we can repeat this process. Note 
that during the process the injection rate of session S has 
decreased from its maximum rate to zero. Once the session 
H fluid has left server Q4 and moved to the next gadget, the 
injection rate of session S increases again to its maximum 
rate. Hence we have created oscillations in the injection rate 
of session S. 

Unfortunately, we cannot continue the above process indef- 
initely. This is because in a finite network the session H has 
finite length and so the session H fluid will eventually reach 
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its destination. We therefore need a mechanism to replenish 
the session H fluid. We do this as foflows. 

At the beginning of the row of gadgets we have a server 
caUed a replenishing server. Initially all the servers in the row 
gadgets are empty and so session H fluid is injected at its 
maximum rate. We are able to create a buildup of fluid in the 
replenishing server This causes session H fluid to build up 
in the replenishing server and it also causes the session H 
injection rate to go to zero. When enough session H fluid has 
been collected it begins the process of passing through the row 
of gadgets. 

It remains to describe how we create a buildup of fluid in 
the replenishing queue. We do this in the following manner As 
the session H fluid passes through the server Q2 in a gadget, it 
creates a buildup at server Q2- We have a new vertical session 

V that also passes through server Q2- Session V is blocked 
by session H at server Q2 and so we get a buildup of session 

V fluid. Our complete example consists of multiple rows of 
gadgets. Each vertical session V passes through multiple rows 
and multiple copies of Q2- Moreover, the session V fluid gets 
blocked at each copy of Q2- Each session V eventually passes 
through a replenishing queue at the beginning of a row of 
gadgets. Since the session V fluid has been blocked at multiple 
copies of Q2 it enters the replenishing queue at a high rate. 
This causes the required buildup in the replenishing queue. 

The vertical sessions V are depicted in Figure |2] The 
interaction of the vertical sessions with the replenishing server 
is depicted in Figure [3] We remark that each server processes 
data in FIFO order with two exceptions. First, each server Q2 
gives strict priority to session H fluid over session V fluid. 
Second, each replenishing server gives priority to session V 
fluid over session H fluid. As we discuss in Section |Vl] we 
made a step towards relaxing the first requirement: we are able 
to create a (more complex) basic gadget where the servers Q2 
are strictly FIFO. We are unfortunately unable to construct an 
example where all servers are strictly FIFO. In Section IVTl we 
conjecture that this is possible and we briefly discuss why we 
believe this is so. 

III. Basic gadget 

In this section we'll define a basic gadget that will be used 
as a main building block in our construction. This gadget 
consists of four servers called Qi, Q2, Q3 and Q4, and two 
sessions. We'll describe a phase in which we transfer fluid 
from server Qi to server Q4. During this phase the height 
of server Q2 will increase and will then decrease again. The 
behavior of server Q2 will allow us (in the full construction) 
to eventually accumulate new fluid that we use to replenish 
fluid that eventually reaches its destination. 



A. Basic gadget: construction 

The gadget consists of the four servers Qi . . . Q4, and two 
sessions: one horizontal session that we call H, and one simple 
session that we call S. The horizontal session H starts and 
ends outside the gadget; in fact, in the full construction one 
such session comes through multiple gadgets. For the simple 
session 5", the source and the sink lie inside the gadget. 



H enters 



session V 
Q2 Q3 



session V _ 
fluid 

session iT 
fluid 



Q4 



session S 



H leaves 



Fig. 1. A basic gadget. At server Q2, session H has priority over session 
V. At server Q4, sessions S and H are served in FIFO order. 



Fig. 2. The giid-like structure 



Session H enters the gadget at server Qi and leaves it at 
server Q4. The path of H first goes K times through the big 
loop, which consists of servers Qi, Q2, Q3 and Q4 in this 
order The path then goes K' more times through the small 
loop, which consists of server Q4 only. Here K and K' are 
parameters that we'U tune later. The path of the simple session 
S goes through server Q4 only. The two sessions compete in 
server Q4 in FIFO order See Fig. [T]for the composition of a 
single basic gadget. 

Let us briefly preview how the basic gadgets fit into the 
full construction. There we shall have rows of servers that 
consist of many blocks of three servers each. Servers Qi, 
Q2 and constitute one such block; server Q4 will be 
the first server of the next block. The horizontal session H 
goes consecutively through all blocks in a given row (more 
precisely, H enters the next basic gadget right after it leaves 
the previous one; the details are Section llVb . The blocks of 
servers are also organized in columns; there will be vertical 
sessions that go through consecutive servers Q2 in the same 
column. These flows will not mix with the horizontal session 
H since server Q2 will strictly prefer H. This grid-like 
construction is summarized in Fig. |2] 

Now let us go back to the level of a single basic gadget. We 
shall have several parameters (apart from K and K') that we'll 
tune later. For the simple session S, let Eg < 1 be the maximal 
flow rate, and let Sg be the damping parameter. The flow 
control parameters of session H are irrelevant at this point, 
simply because its source is outside of the scope. Moreover, 
throughout this section we'll assume that the injection rate of 
session H into the gadget is 0. Servers Qi and Q4 both have 
service rate 1. Server Q2 has service rate 1 — p2 and server 
Q3 has service rate 1 — /is, for some parameters < p2 < Ms- 
We summarize the parameters in Table HI 

B. Basic gadget: a single time phase 

In this subsection we describe the workings of a single time 
phase where we transfer one unit of session H fluid from 
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Parameter 


Description 


(Es, <5s) 


flow control parameters for session S 


K 


number of big loops for session H 


K' 


number of small loops for session H 


1 — A12 


service rate of server Q2 


1 - At3 


service rate of server Q3 



TABLE I 

Tunable parameters in the basic gadget 



time t 


hi 


h2 


^3 


h4 


remarks 


t = 


1 











fresh fluid only; e(0) = Es 


t G [0,Ti] 


J, 


T 


T 


T 




i G [Ti,r2] 





I 


T 


T 




te[T2,T3] 








I 


T 


Ta = K/{1 - 113); e(T3) > es/2 


t G [T3,Tp] 











> 1 




t = Tp 











1 


processed fluid only; e{Tp) = 



NOTE: we assume that the injection rate of session H is for all t. 
NOTATION; hi{t) is the height of server Qi at time t; e(t) is the injection 
rate of session S at time i; a vertical (horizontal) arrow means that a given 
quantity is strictly increasing (decreasing) with time. 

NOTE: session H arrives at server Q2 at rate at most 1 — ^3 between times 
Ti and T3, and at rate between times T3 and Tp. 

TABLE II 

Basic gadget; main events within a single phase 



server Qi to server Q4. 

Let us define /res/z fluid as the fluid from session H that has 
never come through server Qi within our basic gadget, i.e. as 
the session H fluid that still needs to go K times through the 
big loop. Also, let us define processed fluid as the fluid from 
session H that either has come through our basic gadget, or 
just needs to come through server Q4 exactly once to exit the 
gadget. 

At the beginning of the phase we assume that server Qi 
contains one unit of fresh fluid, and the three other servers 
are empty. The injection rate of session S is at its maximal 
value Eg. Moreover, we assume that throughout the phase, the 
injection rate of session H is zero. We finish with one unit of 
processed fluid at server Q4. 

The behavior of the servers during the phase is summarized 
in Table HI] During the time interval [0,Ti], the height of 
server Qi decreases and the height of servers Q2, Q3 and Q4 
increases. During the time interval [Ti, T2], server Qi is empty, 
the height of server Q2 decreases and the height of servers 
and Q4 increases. During the time interval [T2, T3], servers Qi 
and Q2 are empty,the height of server decreases and the 
height of server Q4 increases. We ensure that = K/ (1— /X3) 
and that eiT^) > 6^/2. 

We state our result as follows: 

Lemma III.l Consider the basic gadget with a given flow 
decrease mechanism for session S that satisfies properties 

Assume that the injection rate into session H is at all 
times. Furthermore, suppose that at time i = server Qi 
contains one unit of fresh fluid, the three other servers are 



empty, and the injection rate into session S is at its maximal 
value Eg- 

Then there exist parameters {es,Ss, fi2T K') and times 
(Ti, T2, T3, Tp) such that the gadget functions as shown in 
Table |^ Moreover, this holds for any K > 20, in which case 
we can choose Eg G {5/3K;20/ K) and set fi^ = 2es/5. 

Moreover, it turns out that after the parameters {K,es,fJ.3) 
are chosen in the above fashion, one can pick any fi2 G (0, /^s)- 
This enables us to fine-tune such quantities as T2 and the 
maximal height of server Q2- Although we do not use this 
feature in the present proof, it is useful in the all-FIFO setting; 
see Section [yi] for details. 

C. Basic gadget: proof of the main lemma 

Let hi{t) be the height of server Qi at time t, and let e{t) 
be the injection rate of session S at time t. 

We set ^3 = 2£s/5 and assume < /i2 < /i3. At first 
session H fluid is served by server Qi at rate 1. Since ^2 > 0, 
we start to get a buildup in server Q2- Session H fluid leaves 
server Q2 at rate 1—^2 which is larger than 1 — so we 
start to get a buildup in server Q3, too. Then session H fluid 
leaves server Q3 at rate 1 — /i3 and arrives at server Q4. At 
the beginning of the phase fluid also arrives to server Q4 on 
session S at rate close to Eg. Since 1 — /i3 + > 1 we get a 
buildup at server Q4. 

As server Q4 is congested, e{t) starts to decrease; however, 
we'll make sure that it stays large enough - say, at least e/2 - 
until time T3 when server Q3 empties. Since 1 — fi3+es/2 > 1, 
it follows that (at least) until this time /i4(t) keeps increasing. 
Moreover, until time T3 session H fluid loops back to server 
Qi after being served by Q4 at rate less than 1 — ^3 (since 
Q4 is also serving session S). In particular, hi{t) decreases 
until it becomes 0. 

To summarize, initially the height of server Qi decreases, 
and the heights of the three other servers increase. At some 
time t = Ti, server Qi empties. At this point, the fluid that 
loops back from server Q4 passes directly through server Qi 
and arrives at server Q2 at a rate less than 1 — which is 
less than 1 — /i2- Therefore at time Ti the height of server Q2 
starts to decrease, until at some time i = T2 it empties. At 
this point session H fluid starts to enter server Q3 at a rate 
smaller than 1 — 1^13, so its height starts to decrease. We use T3 
to denote the first time that server Q3 empties. At this point 
all of the session H fluid is stored in server Q4. 

By definition of time T3, at any time t e (0, T3) session H 
fluid leaves server Q3 at the rate 1 — ^^3, so exactly (1 — 113)1 
units of session H fluid are served by Q3 between time and 
t. Since only K units of session H fluid are available (more 
precisely, one unit that passes through the big loop K times), 
it follows that T3 < T, where T = K/{1 - ^3). 

From now on, let us fix parameter K and vary parameter 
Eg. Define constants £1 = 5/3K and £2 = 20/ K. Later we 
will choose £g from the interval [ei;e2]- 

Claim III.2 There exists a positive constant 5* such that 

e{2K) = f5'{£,2K) > e/2 for any e G [£i;e2]. (9) 
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Proof: Let us denote ai ~ Si (3/2)' and let us cover the 
interval [si; £2] with smaller intervals [a^; ai+i]. For each such 
interval let us choose S = Si such that fs{cti, 2K) > ai+i/2; 
such 5 exists by Axiom (AlU since < ai. Then (|9]l 

is satisfied for any 6 > Si and e G [ai;ai+i]. Finally, let us 
choose 5* to be the smallest of the S/s. □ 



For 



Let us set the damping parameter 6s to S* from Claim HIOl 
Note that since ^3 = 2es/5 it follows that T < 2K, so 



> £s/2 for any G [ei;£2]. 



(10) 



We choose parameter £s carefully to ensure that T3 = T, 
or, equivalently, that at time t = all of the session H fluid 
has passed through servers Qi, Q2 and Q3 exactly K times. 
More precisely, we have the following claim whose proof is 
deferred to Section UlI-DI 

Claim III.3 For any K > 20 there exists Ss G {5/3K; 20/ K) 
such that for any ^2 S (0, ^3) we have T3 = K/ (1 — ^3). 

In the remainder of the phase, all of the session H fluid has 
to do is loop through server Q4 another K' + 1 times, which 
takes time at least K' + 1. For any given K (and hence £s, 6s 
and T3), we choose parameter K' to be the smallest integer 
such that by the time + K' the injection rate of session S 
has dropped to zero, and all of the session S fluid has left the 
system. Clearly, such K' exists for any given K. 

The phase ends at some time t = Tp when all of the session 
H fluid has exactly one loop to go in Q4. Obviously, at this 
time all of this fluid is still queued in Q4; since all of the 
session S fluid has already left the system, h4{Tp) = 1 as 
required at the end of the phase. 

To complete the proof of Lemma IIII.ll it remains to prove 
Claim HIO 



D. Basic gadget: proof of Claim \III.3\ 

Let us keep K fixed and assume that £s G [5/3 A'; 20/ K]. 
Recall that e{t) is the injection rate into session H at time t, 
and hi{t) is the height of server Qi at time t. Recall that we 
use Ti to denote the first time that server Qi empties: 

= mm{t > : h,{t) = 0}. 

We will also use di{t) to denote the rate at which session H 
fluid departs from server Qi at time t. 

A key quantity in our analysis is the total amount of session 
H fluid queued in servers Qi, Q2 and Q3. We use h^{t) to 
denote this quantity at time t. We note that /i+ behaves like the 
amount of fluid in a single queue that is fed at rate di{t) and 
has service rate 1-/13. Since £(r3) > Eg/ 2, by the analysis 
in the previous section server Q3 drains after servers Qi and 
Q2. It follows that 

T3 = mm{t > : h+{t)^ 0} 

and, in particular, h^{T3) = 0. 



any time t G [0; T3] the following equations hold: 



hi{t) 
h+{t) 
di{t + hi{t)) 



= 1 - M3 



{d3{t)+e{t) - 1) dt 



(11) 
(12) 



1+ / {di{t) - 1 + fia) dt (13) 
Jo 

1 - 



e{t) + 1 - /i3 



(14) 



Let T = K/{1- /i3) and recall that Ty, < T and e{T) is 
at least £s/2 by We'd Hke to show that T = T'3 if and 
only if h+{T) = 0, and then solve h+{T) = to obtain the 
desired dependency between Es and K. However, under the 
current definition of h^{t) it is trivially for any t > T'3, 
including t ^T. 

To remedy this, let us formally extend equations dl 1H14| | 
to t G [0;2Ar]. More precisely, let us forget the original 
definitions of functions d3{t), di{t), /i4(t) and h^{t), and 
redefine them by these equations; note that these functions 
are uniquely determined by these equations. Obviously, the 
new definitions coincide with the old ones for t G [0,T3]. 

By (|9]l it follows that h^{t) is strictly decreasing for all 
t G [0; 2K], so t ^Ts is its only root. Therefore T = T3 if 
and only if h+{T) ^ 0. By (O, 



h+{T) = 1 - (1 - ns)T + / d^it) dt 



(15) 



Suppose we keep K fixed and vary Eg- Then the integral in (flST l 
becomes a function of Eg', let us denote it by F{es)- This 
function is continuous; this is intuitively clear but nevertheless 
requires a rigorous proof, see Appendix |T] for details. 

Note that we have h+{T) = if and only if F{es) = K-l. 
We'fl show that F{ei) > K - I and ^(£2) < K - I, so by 
continuity of F{es) it will follow that there exists £ G (£1; £2) 
such that F{e) = K - I. 

Let us define a function 



giy) 



1 



and observe that it is strictly decreasing since g'{y) < for 
all y. We claim that 



d^it) G [giEs);g{Es/2)] for any t G [0,T]. 



(16) 



Indeed, since £(t) G [£s/2;£s] for all t G [0,T], it is easy 
to see that h4{t) is strictly increasing. Therefore the function 
x{t) = t + ft.4(t) is continuous and strictly increasing, so for 
any x G [0; T] there exists some t G [0; T] such that x = x{t). 
It follows that 

d4{x) = di{x{t)) = g{E{t)) G [(?(£,); g(£,/2)], 

claim proved. 

Then F{e) G [F-{e): F+{e)], where 

r F-{e) = Txg{E) = K/{l + iE/h) 
\ F+(e) = Txg{E/2) = K/{1+e/10) 
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It follows that 



< 
> 



F- ' 



K+2 
K+1 



< 
> 



K 
K 



\3KJ — " \3KJ 

Therefore by continuity of F{e) for any K > 20 there exists 
some eo e {5/3K; 20/K) such that F(eo) = K - I. Let us 
set Es = £0- 

IV. Full construction 

In this section we'll define the full construction and prove 
that it has the periodicity claimed in Theorem II. 1 1 We'll use 
the basic gadget from Section |lll] as the main building block. 
Recall that this gadget has several tunable parameters (see 
Table |l|i. All gadgets we use will be identical, i.e. will have 
the same parameters. Specifically, we set K = 40 and use 
Lemma fill. II to get parameters {ss, Ss, IJ.2, Hs, K') such that 
£s < 1/2, and the corresponding times (Ti, T2, T3, Tp). 

A. Full construction: the layout 

As we mentioned in Section IIII-AI we organize gadgets in 
rows, so that the same horizontal session H goes consecutively 
through all gadgets in a given row. Formally, a sequence of 
basic gadgets is merged into a row of gadgets as follows. 

Recall that each basic gadget Q contains four servers 
Qi = Qi{Q), and the horizontal session path that we denote 
P{Q)\ recall that this path includes cycles and loops. Let 
QvitQit ■ ■ ,Qn be a sequence of gadgets. Firstly, two new 
nodes s, t are introduced; they designated as the source and the 
sink of the horizontal session H, respectively. Secondly, for 
each i we identify server Qi{Qi) from gadget Qi with server 
Qi{Qi+i) from gadget Qi+i. Thirdly, we define the session 
path for H as 

{s,q)®P{ga)®...®P{gn)®{q\t), 

where q = Qi{Gq) and q' = QiiGii) are two servers, and 
P®P' denotes the concatenation of two paths P and P' such 
that the last node of P is the first node of P' . In particular, 
session H goes through each server Q = Q4{Qi) for the total 
of 2K + K' times: first K times in the big loop of gadget Qi, 
then K' times in the small loop of the same gadget, and then 
another K times in the big loop of gadget Qi+i. 

Several consecutive rows will form a super-row. We will 
have A^sup super-rows, of N^ows each; each row will consist 
of A'cois gadgets. Here A'sup, iVrows and A'cois are parameters 
that we will tune later. Let us number rows and super-rows 
starting from 0, in the downward direction. In a given row, let 
the horizontal session flow from left to right; let us number 
gadgets from left to right, starting from 0. Let G(i^j^k) be the 
fc-th gadget in the j-th row of super-row i. The Arrows gadgets 
Q{i,j,k), j G [-^rows] will form a column Cik- So each super- 
row i consists of iVcois columns Cik, k G [A^'cois]- 

Recall that we have a distinct horizontal session for each 
row. All horizontal sessions have the same flow control 
parameters: we let Eh be the maximal flow rate, and be 
the damping parameter. In addition to horizontal sessions and 
simple sessions from Section |lll] we'll have a new type of 
session: vertical sessions. All such sessions have the same 
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Description 
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flow control parameters for the horizontal sessions 


IIt~\ 

H'U 


service rate for the decelerating servers, fjj ^ 


Nvcrt 


#vertical sessions in each replenishing server 


-^cols 


#columns 


-^rows 


#rows in a super-row 


A^sup 


#super-rows 


At 
AT 

AT* 


time shift between two consecutive rows in a super-row 
time shift between two consecutive super-rows 
duration of a row replenishing process 



TABLE III 

Tunable parameters in the full construction 



flow control parameters: we let be the maximal flow rate, 
and 6v be the damping parameter 

Each server Qi in the very first gadget of each row is called 
a replenishing server. Each vertical session V goes through 
exactly one such server, call it Qy. It will be the case that 
exactly Advert vertical sessions go through each replenishing 
server, periodically flooding it with vertical fluid, causing 
build-up (replenishing) of the blocked horizontal fluid. See 
Section IIV-HI for more details. 

For each replenishing server Q there is a distinct new server 
with service rate /zd > -Avert £v that we call the decelerating 
server. For each vertical session V such that Q ~ Vq this is 
the last server that V goes through before it reaches its sink. 
In case there is a build-up of vertical fluid, this server acts as 
a bottleneck, making sure that the build-up persists (and thus 
session V stays blocked) for a sufficiently long time. 

Each vertical session V is associated with some column 
Cik. Session V injected into server Q2 of the first gadget of 
this column, and goes consecutively through servers Q2 of all 
gadgets in the column. Then V goes through the corresponding 
replenishing server Qy, then it goes through the corresponding 
decelerating server, and then it proceeds to its sink. 

At most one vertical session passes through a given column. 
Some columns do not contain a vertical session; those are 
called blank. We need to specify which columns are blank, and 
which vertical sessions connect to which replenishing queues. 
Let us denote this information as a collection Cprof of all 
4-tuples {i,k,l,j) such that column Cik contains a vertical 
session V, and the corresponding replenishing server Qy lies 
in row j of super-row I. Collection Cpiof describes connectivity 
between the super-rows; thus we call it connectivity profile. 

We review our construction in Figure [3] We will specify the 
connectivity profile in Section IIV-HI We also need to choose 
the values for various parameters, see Table HU] for a list. We 
do it later in this section. 

B. Full construction: the max-stable flow constraints 

We say that at a given time a session S is max-stable if it is 
stable and is injecting at its maximal injection rate. Let us say 
that a super-row 7^ is in a max-stable state if all horizontal 
sessions in TZ are in a max-stable state, and all vertical sessions 
in the columns of TZ are in a max-stable state. To ensure 
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vertical session 
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vertical session 
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replenishing servers 
Fig. 3. Super-rows for Arrows = 4 and Advert = 2. 

that such state is possible, we need to satisfy the max-stable 
flow constraints: for each server Q, the sum of the maximal 
injection rates of all sessions served by Q is no more than this 
server's service rate. Let us list these constraints: 

Ke\i + Sv-^vcrt < 1 (at each replenishing server Qi) 

(2K + K') Eh + Es < 1 (at each non-replenishing Qi) 

Ke\i + Ev < 1 — M2, (at each server Q2) 

Keh < 1 — /i3. (at each server Q3) 

Advert £v < yUD (at cach decelerating server) 

To satisfy these constraints, let us set 

Advert = [l/(/i3-^2)l 
el = l/4iVvcrt 



and let us enforce that 



< £h and Sy < e* and > 1/4- 



(17) 



Since we chose Eg, the maximal flow rate for a simple session, 
to be at most 1 /2, il7\ implies all max-stable flow constraints. 

C. Full construction: the replenishing process 

In this subsection we will consider the dynamics of our 
construction. We will build on the dynamics of the basic time 
phase described in Section |lll] In particular, in a given row 
this process will happen in each gadget, one gadget at a time, 
consecutively from left to right. Different rows will follow the 
same process, shifted in time by a certain amount. 



Let us introduce some machinery needed to describe and 
reason about such processes. Let us start by defining formally 
what is a state in our construction. The state of session 5 at a 
given time t is a record {Q,t, . . .) recording the injection rate 
of S at time t, whether at time t the session is blocked or happy 
or stable, and (if it has a positive injection rate and it is not 
stable) the last time it switched between these three modes. 
The state of server Q at a given time t is a set of records 
{Q, t,S, . . .) which for each session 5* coming through server 
Q describes the session S fluid buffered at Q at time t. 

Let J\f be the set of all servers and sessions. For each subset 
Af' C M the state of A/"' is the union of states of all elements 
of Af' ■ Note that the state of Af at time tg determines the state 
of Af at any future time t > to. We will also argue about the 
dependencies between the states of different rows and super- 
rows in our construction. Finally, let us define the process in 
Af' C A/^ in time interval / as the union of states of A/' for 
all times t E I. 

Now we can introduce a more concrete definition of the 
basic time phase. In a gadget, the B{t)-process is the basic 
time phase which starts at time t and ends at time t + Tp. 
We will ensure that each gadget G{ij.k) undergoes the R{t)- 
process for each 



t = {nNsup + i) aT + jAf + kTp, 71 e N, 



(18) 



where parameters aT and At are the same for all gadgets. 
Each gadget will undergo exactly one i3(t)-process every 
A^sup aT time units. The rest of the time it will be "idle"; 
we'll make it more precise later In particular, all gadgets in 
the same row will follow the same process, but the processes in 
two consecutive gadgets are shifted by time Tp. Similarly, all 
rows in the same super-row will follow the same process, but 
the processes in two consecutive rows are shifted by time At; 
all super-rows will follow the same process, but the processes 
in two consecutive super-row are shifted by time aT. 

Recall that we fixed the value for Tp when we invoked 
Lemma llII.il We want to define At such that < At < T2 and 
the ratio Tp/At is an odd integer; we will use these properties 
in the proof of Lemma IIV.8I to enable equation ( |37] |. Let us 
set 

At Tp/r, where r := 2\Tp/T2] + 1. (19) 

We will specify aT later in this subsection. 

We introduced rows of gadgets in order to enable the 
following intuitive claim: 

Claim IV.l If for time t some gadget in a row undergoes the 
R{t)-process, and the next gadget in this row is 'nice', then 
this next gadget undergoes the B(t + Tp)-process. 

Here the next gadget being 'nice' means that in this gadget, 
servers Q2, Q3 and Q4 are empty of horizontal and simple 
sessions' fluid, and the simple session is max-stable. We will 
need to use this intuitive claim in the subsequent proofs about 
states of the full construction. For this purpoce we need to 
express what happens in this lemma in terms of states, and to 
do that we need a somewhat more complicated formalism. 

To define precisely what we mean by the state of row TZ, 
let us think of 7?. as a system that contains the horizontal 
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Fig. 5. Illustraion of an Sk-state 



session and (for each gadget) the four servers Qi and the 
simple session. Let us define the partial state of row TZ as the 
subset of its state, namely all of the state except the records 
about vertical sessions coming through TZ. Let us define some 
partial states that will be very useful. 

In row TZ, let Qk be gadget k in this row. Say at a given 
time row 7?. is in a Pk-state where < fc < A^cois if 

• all servers in row TZ are empty of horizontal or simple 
sessions' fluid, except server Qi in gadget Qk- This server 
holds no simple session's fluid, and exactly one unit of 
horizontal fluid which is fresh with respect to Qk- 

• the injection rate into the horizontal session is 0; all 
simple sessions in gadgets Qi, I > k are max-steady; 
if fc > 1 then the simple session in gadget Qk-i has zero 
injection rate. (See Figure |4]) 

Say row 7?. is in a Pk-state, fc = Nco\s, if 

• all servers in row TZ are empty of horizontal or simple 
sessions' fluid, except server Q4 in gadget Qk- This 
server holds exactly one unit of horizontal fluid which 
is processed with respect to Qk- 

• the injection rate into the horizontal session is 0; the 
simple session in gadget Qk has zero injection rate. 

Note that Po-state and Pi-states can be seen as a unique 
partial states of row TZ- For fc > 2 we view the Pfc-state 
as a collection of all partial states that satisfy the appropriate 
conditions. Say that a row is empty if all servers in this row 
are empty of horizontal or simple sessions' fluid; let Pcmpty 
be the collection of all partial states of a row such that it is 
empty. The following lemma characterizes the partial states 
that occur after the row is in partial state Pi. 

Claim IV.2 Consider a row TZ in our construction. For con- 
venience, let us denote = A^cois Tp + 1. 

(a) Suppose at time t row TZ is in a Pk-state, for some k 
such that 1 < k < iVcois- Then row TZ undergoes the 
R{t)-process, and at time t + Tp it is in a Pk+i-state. 

(b) Suppose at time Tp row TZ is in the partial state Pi. Then 
for each time t G \Tp,ts,] the partial state of this row is 



fixed, call it P{t). In particular, P{kTp) G Pk for each 
fc such that 1 < fc < A'cois, i^nd P(t») G Pompty 
(c) Suppose at some time t G [Tp, t,] row TZ is in the partial 
state P{t). Then for each time t' G [t, t,] the partial state 
of this row is P{t'). 

In fact each row will be initialized in partial state Po, but in 
this case the behavior of (the partial state of) the row depends 
on what happens outside this row; this is why we chose partial 
state Pi as the starting point for P{t) 

Claim IV.3 Suppose at time row TZ is in partial state Pq, 
and that during time interval [0; Tp] no vertical fluid enters 
the replenishing server of TZ. Then this row undergoes the 
^{Q)-process, and at time Tp it is in partial state Pi. 

Let us say that a given super-row i includes the correspond- 
ing rows, decelerating servers in these rows, and all vertical 
sessions Vik. Let us define the partial state of this super-row 
as the partial state of all rows. For each fc < A'cois such that 



fc Tp - Arrows At > Tp 



(20) 



let us define a partial state such that each row j is in partial 
state P(fcTp - jM). (See Figure |5]) 

Let us translate Claim lrv!2] into the corresponding claim into 
partial states of a super-row. For convenience let us introduce 
the following conventions: 
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iVeolsTp + 1 
+ Throws /r] 

fcoTp. 



(21) 



Claim IV.4 Consider a super-row i in our construction. 

(a) Suppose at some time t super-row i is in partial state Sk, 
for some k such that kg < k < A'cois- Then at time t + Tp 
it is in partial state Sfe+i. 

(b) Suppose at time fco Tp a given super-row is in partial state 
Sfe„. Then for each time t G [Iq] t^] the partial state of 
this super-row is fixed, call it S{t). Moreover, it is the case 
that S{kTp) ~ Sk for all fc such that kg < k < A'cois- 

(c) Suppose at some time t G [to; t^] a given super-row i is 
in partial state S{t). Then for each time t' G [i, i*] this 
super-row is in partial state S(t'). 

Let us say that a partial process in a super-row in time 
interval / is a union of all partial states in /. The above claim 
defines the partial process 



{S(t) ■- t G [to; U]} 



(22) 



For each non-blank column Cik, let Vik be the vertical session 
that goes through Cik- For each such session, we would like 
to understand how it interacts with partial process (|22] | in 
super-row i. Intuitively, for each row TZ of this super-row all 
interesting interaction with session Vik happens when row TZ is 
between partial states P^ and Pfc+i- For technical convenience 
we want partial process (l22l i to be well-defined whenever any 
row is between these two partial states. Therefore we will 
enforce that for any vertical session Vik we have 



fco < fc < iVcols - fco 



(23) 
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In partial process (l22l l all horizontal fluid eventually drains 
to its sinks. As we want our construction to exhibit oscillations, 
we need a way to get back to partial state Sk„ - Let us consider 
this problem for each row separately: we start with an empty 
row, and we want to get back to partial state Pi; most 
crucially, we need to replenish the horizontal fluid in server 
Qi in column 0. 

Before we can state our basic result on replenishing, we 
need to build some machinery. Say row TZ is max-stable if the 
horizontal session and all simple sessions are max-stable, and 
all servers contain no horizontal or vertical fluid. We would 
like to define a process that turns row TZ from a max-stable 
state to partial state Pi; we will call it a row replenishing 
process. Apart from partial state Pi, we need to ensure that 
row TZ behaves 'nicely' towards vertical sessions that come 
through TZ. Specifically, say row TZ is open if for each non- 
blank column in this row, server Q2 (in the gadget in this 
column) has at least Ev units of capacity for the vertical session 
that flows through this column. 

Let us formulate the initial conditions for our replenishing 
process. Recall that each vertical session V goes through some 
replenishing server Qy. For a given replenishing server Q in 
row TZ, let us define the inverse mapping Vq as the set of all 
vertical sessions V such that Q = Qy. Say our construction 
is Q-synchronized if at time t the following conditions hold: 

Row TZ is max-stable; for each session Vik G Vq 
Vik is max-stable and satisfies ( |23] |. (24) 
and super-row i is in partial state Sfc. 

The main result of this subsection is (essentially) that if our 
construction is Q-synchronized, then a certain time later row 
TZ is in partial state Pi. 

Lemma IV.5 There exist parameters (ch, (^h, ^v, (^v, Md) and 
positive integers (Li,L2) such that the max-stable flow con- 
straints ( 1771 ) hold, and for N-^ows ~ r Li and any choice of 
(aT, iVcois, -/Vsup, Cprof ) we have the following property: 
(*) Let Q be a replenishing server such that \Vq\ = Advert- 
Suppose our construction is Q-synchronized at time t. 
Then at time t-\-^T* server Q's row is in partial state Pi, 
and each session Vik G Vq is happy. Moreover, between 
time t and t + aT* this row is open. Here 

aT* = (Li +L2 + 1)7> - A<. (25) 

We defer the proof to Section IIV-DI We would like to 
make several remarks. First, note that property (*) is con- 
ditional: in this lemma we do not prove that our construc- 
tion ever becomes Q-synchronized. Second, we achieved 
this property by tuning parameters (eh, <^h, Ev, <5v, Md, ^rows)- 
Note that we did not impose any constraints on parameters 
(aT, A/'sup, -/Vcois, Cprof )• Third, we had a lot of flexibility in 
choosing the separation time aT*. We used this flexibility to 
enforce ( 1251 ), which we will use in order to make our construc- 
tion Q-synchronized at the appropriate times. Specifically, we 
will use ( l25T l in the proof of Lemma I1V.8I in order to pass 
from (35[ to ( l36b . 

Let us conclude this subsection by attaching appropriate 
names to the process described in Lemma HV. 5 1 Let us say that 



a {Q ^t) -replenishing process is (any) process that happens in 
our construction between times t and t + aT* if at time t 
it becomes Q-synchronized. By Lemma |IV.5I during a {Q,t)- 
replenishing process the corresponding row goes from being 
empty at time t to partial state Pi at time t + aT*. Informally, 
we say that this row gets replenished. 

Let us define the analogous replenishing process for a given 
super-row i. Let Q^^' be the replenishing server in row j of this 
super-row, so that all rows experience the same replenishing 
process, appropriately time-shifted. Let us say that a {i,t)- 
replenishing process is (any) process in our construction that 
starts at time t and proceeds for time (Arrows — l)At + aT*, 
such that for each row j in super-row i, a (Q*-^-*, t + j At)- 
replenishing process happens. 

D. Full construction: proof of Lemma \IV.5\ 

Consider a session with maximal flow rate e and damping 
parameter 6. Recall that fs is the parameterized flow decrease 
function (see Section II-BI) . Suppose at time the session is 
injecting at rate e, then becomes blocked and stays blocked. 
Then by axiom (AO the injection rate starts decreasing until 
at some finite time Tstop ~ Tltople, ^) it becomes 0. Let 

Aie,6,t) := / fsie,t)dt 
Jo 

be the amount of fluid that is injected by time t. Let 

B{e,5) / fs{e,t) dt ^ A{e, 6,T,,,p) 
Jo 

be the amount of fluid that is injected by the time the flow 
stops. Let By — B{ev,6v) and B^ — i3(eh,'^h) be these 
amounts for vertical and horizontal sessions, respectively. 

Let us fix a choice of (A"cois, ^sup, Cprof ), suppose Q is a 
replenishing server such that with respect to this choice | Vq | = 
Avert, and assume that our construction is Q-synchronized at 
time t. Let us focus on some vertical session V = Vik £ 
Vq. The path of this session goes through Arrows consecutive 
gadgets in column Cik. Let Qj = Q{i,j.k} be the j-th such 
gadget, and let Q^^^f be the server Q2 from Qj. Say such server 
is blocked if it stores a positive amount of horizontal fluid. 
Then, since at servers Q^-*^ the horizontal fluid has priority 
over the vertical fluid, session V is blocked if at least one 
server Q(J) is blocked. 

Recall that at time t super-row i is in partial state Sfc. Let 
us consider what happens with session V after time t. Until 
time t session V is in the max-stable state. Then each server 
Q'^-'^ is blocked starting from time t + j At for time T2, so 
session V is blocked starting from time t for time 

t* := (Arrows -1) At + r2. (26) 

We will make sure that by this time the injection rate into 
session V goes from Sv to 0: 

r.top(ev,<Sv) (27) 

Then By units of session V fluid is accumulated in server Q 
at time t + t* . From this time on, all servers Q''-* become 
unblocked; in each of them, horizontal fluid eats up at most 
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1 — 1^13 units of bandwidth (this is by Lemma IIII.ll ). so at 
least — fi2 units of bandwidth are available to session V. 
Therefore, starting from time t + t* session V fluid drains 
down into server Q at rate at least /is — 1^2, until at some time 
t2 all By units of fluid are gone. 

Since this happens for all A'vcrt vertical sessions 1/ e Vg, 
between times t + t* and t2 the total incoming rate of vertical 
sessions into server Q will be at least Advert (M3 — which is 
at least 1 by definition of Advert - Therefore at time t+t* vertical 
fluid will immediately start building up in server Q. Since at 
Q the vertical fluid has priority over the horizontal fluid, this 
build-up will persist exactly for time Advert By. During all this 
time vertical fluid exits server Q and enters the corresponding 
decelerating server Dq at rate 1. We will choose the service 
rate for Dq so that /^d < 1- Therefore after leaving server 
Q, vertical fluid builds up in server Dy and, moreover, this 
build-up drains down to the sink only after all vertical session 
fluid drains down from Q. Therefore a total of A'vcrt By units 
of vertical fluid drains down server Dy, at a constant rate 
/iD, taking time A'vort -Bv/a'd- During this time interval, the 
injection rate into each session V E Vq is 0. We will ensure 
that this time interval is sufficiently long, namely that 

Advert Sv/md = iVvcrt By + Tp. (28) 

Note that if holds then /^d < 1- 

Let H be the horizontal session coming through server Q. 
Before time t + t* session H is in the max-stable state. Recall 
that server Q holds a non-zero amount of vertical fluid starting 
from time t + t*, for time A'vcrt By. During this time interval, 
session H is blocked. We will tune the parameters so that 
during this time interval the injection rate of session H goes 
from £h down to 0: 

Tstop(eh, 4) < A'vcrt Sv, (29) 

and meanwhile exactly 1 unit of session H fluid accumulates 
in server Q: 

Sh - 1. (30) 

If equations ( l28ll30l l hold, then at time t + t* + Nycrt By the 
row TZ containing server Q in partial state Pq. By Claim HV3] 
this row is in partial state Pi time Tp later, i.e. at time t + AT* 
where 

AT* =<*+ Advert Bv + Tp. (31) 

Moreover, by (l28l l at this time the decelerating server of row TZ 
is empty, and therefore all vertical sessions G Vg are happy. 
Row TZ is open between time t and t + aT* since there was 
no build-up of horizontal fluid at time t, and the horizontal 
session stayed blocked between time t and t + aT*. 

To complete the proof of Lemma IIV.51 it remains to 
match dlB with the definition of aT*. 

Note that by definition (|26] | of t* and since 

7V„ws At = Li \Tp/T2^ At = Li Tp, 
we can write t* as a function of Li. 

t* = Li Tp + Ta - At. 



In particular, by ( l3Tl i equation (l25T l becomes equivalent to 

Ny„tBy^L2Tp-T2. (32) 

Now to prove Lemma IIV.51 it remains to choose param- 
eters (eii, (5h, £v, (^v, /io) and integers (Li,L2) so that equa- 
tions O, (l27H30b and ^ hold. We need to be careful in 
order to avoid circular dependencies between the parameters. 
This is how we overcome this hurdle: 

1. choose (ch, so that Eh < Ei* and = 1. 

2. define 

L2 := 1 + r(T„op(eh, <5h) + T2)/Tp'] 

To (LaTp-Ta) /A'vcrt. 

3. choose {ev,5v) so that Ey, < e* and By = Tq. 

4. choose Li large enough so that (|27] | holds. 

5. choose /iD so that (|28] | holds. 

Note that at the third step we ensure that ( |29] l and ( |32] | hold. 
At the fifth step, by (l32]i we have A'vcrt By > Tp, so /iD > 1/2 
as required. It remains to show that we can indeed do the first 
and the third step. We establish this via the following claim; 

Claim IV.6 If axioms hold, then for any given posi- 

tive (e*, b) there exist flow control parameters (e, 5) such that 
e < e* and B{e, S) = b. 

Proof Choose parameter d so that B{e*,S) > b, i.e. 
at least b units of fluid are injected if the session becomes 
blocked when it is transmitting at rate e*; such 5 exists by 
axiom (AlHi. Now that 5 is fixed, let us tune e in the interval 
(0, £*] so that exactly b units of fluid are injected. We do it 
rigorously as follows. 

Let us write rstop(e) — Tstop(£,(5); by Axiom (A|3]l this is 
an increasing function of e. The function 

A{e) := A{e, S, T,,op(e*)) 

is continuous by axiom (A|5]l, see Fact 11.41 for rigor. Since 

A{0) = and A{e*) > b, we can choose e < s* such that 
A{e) = b. Since T,t„p{e) < T,top{e*), it follows that B{e,S) = 
b, too. □ 

This completes the proof of Lemma IIV.51 

E. Full construction: the connectivity profile 

In this subsection we will choose the connectivity profile 
Cprof- We will assume that the parameters are chosen as per 
Lemma |IV5] and that aT = 2aT*. 

Recall that Cprof is the collection of all 4-tuples {i,k,l,j) 
such that vertical session Vik goes through the replenishing 
server which lies in row j of super-row I. Let us impose some 
simple conditions on the connectivity profile. 

Definition IV.7 Let us say the connectivity profile is well- 
formed ;/ it has all of the following properties: 

(a) for each pair (i, k) there is at most one {i,k,l, j) G Cprof- 

(b) for each tuple (i, k, G Cprof have fc > fco + 1. 

(c) for each replenishing server Q we have \Vq\ = A'vcrt- 
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(d) Cprof is not changed by a circular permutation of super- 
rows: (z, fc, Z, i) e Cprof if and only if (0, k, I* , j) G Cpiof, 
for l* = {l~ j) (mod iV.up). 

Let us reverse-engineer some other useful properties of Cprof 
from the requirement that for each replenishing queue Q, our 
construction needs to be Q-synchronized at the appropriate 
time. Let us go back to (fTSl i: this is when we would like a 
given gadget to undergo a B(t) process. It follows that for 
each column Cik we would like the corresponding super-row 
i to be in partial state at time 



Tik 



i/^T + k Tp 



Similarly, for each replenishing server Q in row j of super-row 
i we would like this row to be in partial state Pi at time 

tq := i aT + jAi + Tp. 

It follows that we would like our construction to be Q 
synchronized at time tq — aT*. In particular, for each vertical 
session Vik £ Vq we want 



T,k =TQ- AT* (mod Nsnp AT) . 



(33) 



Say the Cprof is synchronized if (l33T l holds for each replenish- 
ing queue Q and each vertical session Vik G Vq. 

Note that a well-formed connectivity profile Cprof is com- 
pletely determined by A'sup and the set C*^.^f of triples {k,l, j) 
such that {Q,k,l,j) g Cprof- Let us call this set the reduced 
connectivity profile. 

For a given reduced connectivity profile 0*^.^^ let us define 

iV,,p(C;rof) := max{Z £ N : (fc, l,j) £ C;rof} 
^cois(C;rof) := max{fc G N : (fc, Ij) G C;rof} 

to be, respectively, is the maximal super-row that super-row 
is connected to, and the maximal non-blank column. Then 
we can choose any A'sup > ^sup (Cprof) reverse-engineer 
the corresponding connectivity profile Cprof- Note that if it is 
well-formed then we satisfy (|23] | if and only if 



A^cols>iVcols(C*of)+il + l- 



(34) 



Clearly, being well-formed and being synchronized are 
properties of C*,.Qf. More precisely, for a given C*,.Qf the 
connectivity profile is well-formed (resp. synchronized) for 
some A'sup > ^sup{C*^^f) then it is well-formed (resp. 
synchronized) for all such A'sup; in this case say that C*j.^f is 
well-formed (resp. synchronized). It turns out that these two 
properties are all we need from C*j.^f. 

Lemma IV.8 // aT = 2aT*, where aT* is given by 
then there exists a reduced connectivity profile C*j,Qf which is 
well-formed and synchronized. 

Proof: Recall that within a given super-row, rows are 
numbered from to Arrows — 1- Let us write row numbers as 
ir + j, where j G [r]. Note that C*j.Qf is synchronized if and 
only if for each triple (fc, /, ir + j) G C^.^^^ we have 



Indeed, this matches (l33T l since tqa: = kTp and the right-hand 
side of (|35] | is tq, where Q is the replenishing server in row 
ir+j of super-row /. Let L = L1+L2. Then by equations (fT9] l 
and (IZST i we can rewrite ( l35T l as follows: 



kr = {21 - l)Lr + {21 + i) r - {21 - 1 - j). 
Equivalently, for some a G N we have 



21 = ar +j + 1 

k = L{ar + j) + a{r-l) + {i+j + l). 



(36) 



(37) 



k Tp + aT* =lAT+{ir + j) At + Tp. 



(35) 



Note that in the above equation we want I and k to be integer; 
this is why in (fT9] l we defined r to be integer. Also, note that 
the first equation in (|37] | cannot hold for odd j if r is even; 
this is why in ( fT9] l we defined r to be odd. 

Let Aj be the set of the first A'vcrt integers that are at least 
3 and have the same parity as 1 + j. Let us define C*j.^f as 
the collection of all triples {k,l,ir + j) such that a G Aj, 
i G [Ti], j G [r], and (fc, I) are given by (l37T i. 

Since r is odd, such C*^.^^ is synchronized. Let us check that 
C*j.Qf is well-formed. We need to check the four properties in 
Definition IIV.7I Properties (b) and (d) are easy: the former 
holds since the as are at least 3, and the latter holds since 
we reverse-engineer Cprof from C*^,^^. 

To prove property (a), let us consider equation i35[ . By (IZSl l 
we have aT > A'rows At, so for a given choice of k there can 
be at most one value I such that (l35l l holds, hence at most one 
triple {k, I, ir + j) G C*^af Therefore property (a) holds. 

It remains to check property (c), i.e. that for each replen- 
ishing server Q we have \Vq\ = N^cn- Indeed, say server Q 
lies in row ir + j of super-row /; for simplicity let us assume 
I > A'sup (CpjQf)- Then the induced connectivity profile Cprof 
contains a tuple (n, k, n' , ir+j) if and only if for some a G Aj 
the pair (Z, k) satisfies dJTl i where the super-row number I is 

I = {n' - n) (mod A'^^up)- 

Claim follows since there are exactly A'vcrt such pairs. □ 



F. Full construction: safe states 

Consider a given vertical session Vik. Suppose super-row 
i is in partial state S(to) at time r^o + to- As far as we are 
concerned. Lemma IIV.5I specifies the behavior of session Vik 
between times Tik and Tik + aT*. Here we investigate what 
happens with Vik during the rest of the time interval [tp; t*\, 
and also when super-row i is empty. 

Suppose a vertical session V comes through a server Q. Let 
us say that at a given time server Q is open for session V if 
Q has at least units of capacity available for V . Suppose a 
given super-row contains session V . Let us say that at a given 
time this super-row is open for session V if each server in this 
super-row is open for V. 

Say a vertical session V is safe at a given time if it enters 
the corresponding replenishing server Qv at rate at most e^. 
Say a replenishing server Q is safe at a given time if it is 
empty of vertical fluid, and all vertical sessions G Vq are 
safe. Say a super-row is safe if each replenishing server in this 
super-row is safe. Intuitively, if vertical sessions, replenishing 
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servers and super-rows are safe then they do not cause any 
perturbations in our construction. 

Let r/j, := Tifc + (iV,.ows - l)Ai. 

Lemma IV.9 Consider a vertical session Vik- Suppose for 
some t G [to; t^] at time Tio + t super-row i is in partial 
state S(t) and session Vik is max-stable. Then 

(a) session Vik is safe during the time interval(s) 

[no +t; Tio+ t,] \ [r-f. ; Tik + aT*] , 

(b) super-row i is open for Vik during the time interval(s) 

[no + t; no + 1*] \ [nk ; nfJ\ . 

(c) ;/ at time r^o + super-row i is safe and session Vik 
becomes unsafe at a later time t', then super-row i must 
have become unsafe during time interval (t,;o + i*; t'). 

Proof The proof of parts (ab) is in-lined in the proof of 
Lemma I1V.5I For part (c), note that at time both super-row 
i and session Vik are safe, and by ( l23T l there is no build-up 
of horizontal fluid in columns fc' < fc of super-row i. Session 
Vik can become unsafe only (a positive time after) some build- 
up of horizontal fluid appears in some row TZ in column fc. 
The latter can happen only if at some earlier time, in row 
TZ the horizontal session is injecting at the positive rate, then 
becomes blocked by vertical fluid in its replenishing server Q. 
The latter can happen only after server Q becomes unsafe. □ 

Say a super-row is empty at a given time if all rows 
in this super-row are empty. We will also need a version 
of Lemma |lV.9f c) where initially super-row i is empty, as 
opposed to being in one of the partial states S(t). 

Lemma IV.IO Consider a vertical session Vik- Suppose at 
some time t super-row i is safe and empty, and session Vik 
is safe. If session Vik becomes unsafe at a later time t' , then 
super-row i must have become unsafe in time interval [t; t'). 

The proof follows that of Lemma llV.9r c). The way we are 
going to apply Lemma llV.9t c) and Lemma HV. 101 it is crucial 
that super-row i becomes unsafe strictly before time t' . 

G. Full construction: the high-level layout 

In this subsection we complete the specification of our 
construction. We choose parameters (sh, ^h, ^v, (5v, Md, ^rows) 
and positive integers {Li,L2) from Lemma [lV.51 We set 
aT = 2 aT*, where aT* is defined by We choose C*^„f 
from Lemma I1V.8I We define A'cois to be the smallest integer 
such that ( l34l i holds and 

aT divides iVcois Tp. (38) 

Such TVcois exists because Li aT is divisible by Tp by dZSl l. 
We need dSSl l for technical convenience, so that in later proofs 
we could consider moments of time when every super-row is 
either empty or in some well-defined partial state S(t). 

We choose A^sup > Nbup{C*^oi) l^ge enough so that all 
sessions are in the happy state long enough to build up the 
injection rate to the maximal value. Specifically, we let 

iVsup = A^coisTp/AT + ri + l, 



where n is the smallest integer such that if a horizontal session 
and a vertical session and a simple session are in a happy 
state from time onward, then by time n aT they will be 
injecting at their respective maximal rates. This completes the 
construction. 

Let us check that A^sup > -^sup(C*,.of), as required by 
definition of Cprof*- Indeed, let A'^5up(C*^Qf ) = / so that 
(fc,^, •) £ C*j.Qf for some fc. Since the connectivity profile is 
synchronized, (l35T l holds. It follows that 

iVsup-l>fcTp/AT>/-l, 

claim proved. 

H. Full construction: the high-level dynamics 

In this subsection we put all pieces together and show that 
starting from a suitable initial state our construction exhibits 
oscillations, with period Tyg ~ Ngup aT. To distinguish it 
from the 'local' period Tp, let us call Tbig the big period. 

For n E N, define i„ = n (mod A'sup). Recall the 
conventions (l2Tl i. We want our system to have the following 
behavior: 

For each n e N, at time n aT + fco Tp 
super-row i„ is in partial state S^^. 

Moreover, we want super-row i„ to get to partial state Sko as 
a result of an (i, t)-replenishing process, for the appropriately 
chosen time t. 

If ^9} holds, then by Claim IIV.4] it is the case that at each 
time t g nAT+ [koTp,t^] super-row i„ is in state S(t), so 
that S{t^,) S Sempty Morcovcr, we want this super-row to 
stay empty (almost) till time n aT + Tbig. 

We want to prove oscillations via an inductive argument. 
Generally, for some time to we want to define some state H„ 
such that at time to + n aT we want our construction to be in 
state H„; for a given n this will be our inductive hypothesis. 
The sequence {H„ : n G N} should have period Tbig. We need 
to prove the inductive step from n = in to n = m + 1. Then 
we get oscillations if at time to we initialize our construction 
in state Hq. 

With the above plan in mind, we have three difficulties to 
overcome. First, to define any reasonable H„ we need to define 
states of super-rows, not partial states. For this we need to 
specify exactly the behavior of the vertical sessions so that 
this behavior is consistent across different super-rows; it is 
somewhat non-trivial to specify this behavior in a brute-force 
way. Second, we know how to argue about partial states S{t), 
so in H„ we would like each super-row to be in one of these 
partial states. For example, if (|39] | holds for n = 0, then 
the earliest time super-row is in one of these partial states 
(namely, Hfc„) is time fcg Tp. Third, for technical convenience 
in a given inductive step we want to worry only about a single 
replenishing process, namely the one for super-row i„. In 
particular, we would like it to be the case that at time to+n aT 
the corresponding replenishing process for super-row i„ + 1 
has not yet started. These considerations motivate the choice 
of to := ko Tp and aT := 2 aT*. 

For a given super-row, we are going to define a process 
{S*(t) : t > to} such that state S*{t) induces partial state 
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S{t) for any time t G [to,^*]- We want this process to be 
consistent with the behavior of our construction if i39[ holds 
for all n ^ N. To achieve this, we will modify a super-row 
so that it becomes a (suitable) stand-alone system, and in this 
system we will define a process by specifying the (suitable) 
initial state at time to and letting the system run. We let S*(t) 
be the state of this system at time t. 

Let us consider a super-row 5 as a stand-alone system that 
includes all vertical sessions coming through the columns of S, 
and all decelerating servers corresponding to the replenishing 
servers located in S. Note that system S does not include 
the vertical sessions that go to these replenishing servers. 
Recall that in the full construction for each vertical session 
V in S there would be a decelerating server D (in a different 
super-row) which session V goes to. Essentially this server 
determines for how long session V remains blocked. Since 
here we are trying to emulate the behavior of a super-row 
in the full construction, let us add such servers artificially. 
Specifically, let us assume that for each vertical session V in 
S there is a distinct new quasi-decelerating server Dy with 
service rate /io /N^crt, such that session V leaves <S, then goes 
to Dy, and then goes to its sink. Then <S is a fully specified 
dynamical system. 

Let us define the initial state Sq for system S such that it is 
in partial state Sfc^ , all replenishing servers are empty, and all 
vertical sessions in S are in the max-stable state. (It follows 
that in state Sq all servers in S are empty of vertical fluid.) 
Suppose we initialize system S at time to in state Sq, and let 
it run according to its control mechanism. Define S*{t) to be 
the state of this system at time t. Note that at time Iq system S 
is in partial state Sfc^, so by Claim HY41 at any time t G [t,t^,] 
it is in partial state S{t), as required. 

Let us go back to our construction and ask how can we 
make sure that if we start a given super-row in state Sq, it will 
indeed follow the process S*{t). Essentially, it will happen as 
long as the super-row is safe. 

Claim IV.ll Suppose super-row i is in state S*{t) at some 
time t > to, and it is safe between time t and time t' > t. 
Then at time t' it is in state S*{t'). 

Now we can formulate our inductive hypothesis, i.e. specify 
what is state H„. We define it by specifying separately the 
state of each super-row: H„ is the state of our construction 
such that each super-row i is in state S*(to + in-i ^T). Note 
that the sequence {H„ : n G N} has period Tbig, as required. 

Finally, we are ready to state and prove the theorem that 
our construction exhibits oscillations. 

Theorem IV.12 Suppose we initialize our construction at time 
to in state Hg. Then at time to + Tbig it is in the same state. 

We formulate the proof as the next subsection. 

/. Full construction: proof of the main theorem 

It suffices to prove the inductive step. Let us assume that 
for some n G N, at time ti ;= to + n aT our construction 
is in state H„, i.e. each super-row i is in state S*{to + 



in-i aT). We need to prove that at time ti + aT it is in 
state H„+i. Equivalently, we need to prove that super-row 
n + 1 is in state S^, and any other super-row i is in state 
S*(to + in-i + aT). In short, we will get the former via 
replenishing processes (one for each row in super-row n + 1), 
and the latter via Claim HV. Ill 

For simplicity let us assume that Nsup{C*^.Qi) < n < Ngup- 
Then at time ti each super-row i < is in state S*(ii — i aT), 
and super-row n + 1 is in state S*{to + {Ngup - 1) aT). Note 
that by the choice of parameter A^sup at time ti all rows in 
super-row n + I are max-stable. 

Let us consider row TZ of super-row n + 1. Let Q and D be 
the corresponding replenishing and decelerating servers, and 
let t2 := tq - aT*. Note that t2 > ti. Indeed, by (l25]l 

AT* > (Li + 1) Tp = to 

t2 > {n + 1) aT + Tp - AT* 

= (nAT + to) + (aT* -<o) +rp 
> ti+ Tp. 

Claim rV.13 At time t2 our construction is Q-synchronized. 

Proof: We need to check the conditions in the defini- 
tion (l24l i. Indeed, by the induction hypothesis at time ti row 
TZ is max-stable, each vertical session Vik G Vq is max- 
stable, and the corresponding super-row i < n is in partial 
state S(ti - i aT). Therefore by Claim IIV.4T c) at time t2 it 
is in partial state S(i2 — i aT). Note that t2 — Tik since the 
connectivity profile Cpiof is synchronized. It follows that at 
time t2 super-row i is in partial state S(Tofc). 

By Lemma |lV.9f a) during time interval [ti; t2] session Vik 
stays safe. Since this happens for all sessions Vik G Vq, during 
this time interval servers Q and D are empty, so row TZ stays 
max-stable. Moreover, by Lemma IIV.9f b) between times ti 
and t2 super-row i stays open for Vi^, so all servers on the 
flow path of Vik are open for Vik, so it stays max-stable. We 
have checked all conditions in definition ( l24l i. □ 

By Lemma HV. 5 1 it follows that at time tq row TZ is in partial 
state P(Tp), each session Vik & Vq is happy, and moreover 
row TZ is open between times t2 and tq . We use this to prove 
the following two claims. 

Claim IV.14 At time ti + aT replenishing server Q and 
decelerating server D are empty, and row TZ is in partial 
state P(to ~ i At), where j is the row number of TZ. 

Proof: At time tq all vertical sessions Vik S Vq are 
happy. It follows that they are safe and servers Q and D are 
empty. By Lemma |lV.9f a) these sessions are safe during time 
interval [tq\ ti + aT]. Therefore at time ti + aT servers Q 
and D are empty. 

Since at time tq row TZ is in partial state P(Tp), by 
Claim IIV.2r c) at time <i + aT this row is in partial state 
P(t), where t = Tp + ti + /^T - tq = to - j At. □ 

Claim rV.15 Row TZ is open between times ti and t + aT. 
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Proof: We already proved that row TZ stays max-stable, 
hence open, between times ti and t2 - By Lemma lTV.Sl it is open 
between times t2 and tq. Finally, at any time t G [tq, ^i+aT] 
this row is in partial state 'P{t'), t' < ig, and therefore it is 
open by ( l23T i. □ 

Since Claim HV. 1 41 holds for all rows in super-row n+1, at 
time t + aT this super-row is in partial state Sk„ ■ We need 
some more work to go from statements about partial states of 
super-rows to statements about their states. 

Claim IV.16 Let V be a vertical session V in super-row 
Then between time ti and ti + aT (a) this super-row is open 
for V, and (b) session V is safe. 

Proof: Part (a) follows by applying Claim HV. 151 to every 
row in super-row n+1. Part (b) follows from part (a) since 
by induction hypothesis at time ti session V is max-stable. □ 

Say a replenishing server is essential if it lies in super- 
row n+1. Say a vertical session V is essential if V G Vq 
for some essential replenishing server Q. We show that non- 
essential vertical sessions and replenishing servers are safe as 
far as we are concerned. 

Claim IV.17 For each non-essential vertical session Vik, in- 
tervals [ti; ti + aT] and [r^fc; T,jt + aT*] are disjoint. 

Proof: Suppose Vik £ Vq for some replenishing server 
Q in row j of super-row I n + 1. If I < n then 

Tik + aT* = / aT + j At < n aT + <o = ^1- 
If l<n + 2 then T,k > (n + 2) aT - aT* > ti + aT. □ 

Claim IV.18 Between time ti and ti + aT, all non-essential 
vertical sessions and replenishing servers are safe. 

Proof: By induction hypothesis this condition holds at 
time ti. Suppose it fails at time t S (ti, ii + aT], and let us 
assume this is the first time it fails in this time interval. 

If a non-essential replenishing server Q becomes unsafe at 
time t, then at some time t' S {ti,t] some vertical session 
V G Vq must have become unsafe. Therefore without loss of 
generality a non-essential vertical session Vik becomes unsafe 
at time t G (ti,ti + aT]. Let us argue to the contradiction. 

By Claim IIV.16f b) session Vik cannot belong to super-row 
n+1. By induction hypothesis and (l38T l, at time ti either super- 
row i is empty or it is in a well-defined partial state S(ti — Tio). 
In the first case by Lemma H V. 1 01 super-row i becomes unsafe 
at some time t' G {ti,t), contradiction. So we are in the second 
case. If t — Tio > then by Lemma |IV.9f c) super-row i 
becomes unsafe at some time t' G {ti,t), contradiction. Else 
by Lemma llV.9f a) we must have t G [rik; Tik + ^T*], which 
contradicts Claim II V. 171 □ 

Claim IV.19 At time t + aT super-row 72+1 is in state Sq. 

Proof: We know that at time t + aT this super-row is 
in partial state S^.^, and (by Claim [TV. 141 ) all replenishing and 
decelerating servers in this super-row are empty. 



It remains to prove that at this time each vertical session 
V in this super-row is max-stable. This is the case because 
it is max-stable at time <i by the induction hypothesis, and 
during time interval [ti; ti + aT] all servers on the flow path 
of V are open for V. Namely, during this time interval the 
corresponding replenishing and decelerating servers are open 
for V by Claim IIV.18I and super-row n + 1 is open for V by 
Claim HvMa). □ 

For each super-row i 7^ n + 1, note that by Claim |lV.16f b) 
and Claim IIV.18I it is safe between time <i and ti + aT. 
At time ti it is in state S*(<o + *n-i aT) by the induction 
hypothesis, so by Claim IIV. 1 1 1 at time ti + aT it is in state 
S*(to + hi-i + aT) as required. 

This completes the proof of Theorem IIV. 121 and therefore 
the proof of Theorem 11.11 

V. Same damping parameter for all sessions 

In this section we fine-tune the main theorem (Theorem lI.il ) 
so that all sessions have the same damping parameter This 
result is stated in the Introduction as Theorem 11.21 Here we 
prove it in a somewhat more general form. 

We consider flow decrease functions of the form 

fsis,t)=e^h{e,S)fit), (40) 
where for any fixed S G S'del we have 

lim h{e,S)/e^oo. (41) 

To make such functions satisfy axioms (A|T1-Al5]l from Sec- 
tion ll-BI let us impose some natural constraints on smoothness 
and monotonicity: 

• f{t) is continuous on [0, 00] and increases from to 00, 

• h{£, S) increases in 5 from to 00, for any fixed e > 0, 
. h{0, S)^0 for aU 6 G S'del, 

• h{e, S) is differentiable in e, for any fixed S G S'del- 

• h{£, S)/e is decreasing in e, for any fixed S G S'del- 
The last of these conditions is motivated by ( l4Tl i- It is included 
specifically to ensure that fs is increasing in e whenever it is 
well-defined, i.e. whenever it is non-negative; one can see it 
easily by differentiating h{e,S)/e and ( l40l i with respect to e. 

If a flow decrease function function satisfies all these 
conditions, let us call it splittable. This definition is motivated 
by the fact that for such functions we will be able to split any 
session with damping parameter S into several parallel sessions 
with any given damping parameter S' < S. An example of such 
functions is given in (O. 

Now we can state our result as follows: 

Theorem V.l Suppose in Theorem\rj]the flow decrease func- 
tion is splittable. Then we can choose the same damping 
parameter for all sessions. Moreover, there exists 6* G 5del 
with the following property: for any S G S'del such that S < 6* 
we can choose the damping parameter to be 5. 

Proof: Recall that in our construction we have three types 
of sessions: simple, horizontal and vertical, with three different 
pairs of flow control parameters (cg, 5s), (sh, <5h) and (sv, <^v)- 
Let 5* be the smallest of the three damping parameters. 
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We will transform our construction so that all sessions have 
damping parameter 6*. (The same transformation also works 
if we replace S* by any given smaller value.) 

Let us focus on one type of sessions. Let (e, 6) be its 
flow control parameters. If S ^ S* then we do not need to 
do anything. Else, we will replace each session of this type 
with several parallel sessions with damping parameter S* and 
maximal injection rates (ei,ei, Sk) that sum up to e, 
where the rates Si are chosen so that 

k 

fs{e,t)=Y,fs.{e,,t) for all times i > 0. (42) 

i=i 

Assume such rates exist; we will prove it later Then if these 
k new sessions all become blocked when they are sending 
at their respective maximal injection rate, they decrease their 
sending rate exactly as the original session did. It is crucial 
that in our construction each session becomes blocked only 
when it is at the maximal injection rate. 

For the modified construction, we choose iVgup, the number 
of super-rows, exactly as before, except now we want it to be 
large enough so that all new sessions have time to build up 
their injection rate to the maximum values. Then the modified 
construction works exactly as the original one. 

It remains to prove that for any pair (e, S) there exist rates 
(ei,ei, Efc) that sum up to e and satisfy (l42l i. Indeed, 
whenever the rates are non-negative and sum up to e, we have 

k k 

Y,fs'{e^,t) = e- f{t)Y,H£^,n■ 

i=l i=l 

If Si = e/k for each i, then 

Y.1^^h{£,,5*) = kh{£i,5*) ^ eh{ei,5*)/£i, 

which is greater than h{£,5)\fk is large enough. Let us choose 
the smallest such fc; note that k ^ 1 since 5* < S. 

Now let us perturb the rates so that the above sum is exactly 
equal to /i(e, 6). With this goal in mind, let us define a one- 
dimensional function 

i=2 ^ ^ 

Then H{£i) is continuous on [0;e], and H{£/k) > h{£,S). 
Moreover, 

1=1 \ ^ 

which is less than or equal to h{£, 5) by the choice of k. 
Therefore there exists £i € [0, e/fc] such that H{£i) = h{£, S), 
as required. For this value of £i we define the other rates 
accordingly: = (e — £i)/{k ~ 1) for each i > 2. This 
completes the proof of the theorem. □ 

VI. Further research: the all-FIFO setting 

We conjecture that Theorem II. 11 extends to the case when 
all servers must be FIFO. We have a promising preliminary 
result in this direction, namely an all-FIFO version of the basic 



gadget in which server Q2 is strictly FIFO and does not give 
session H priority over session V. 

We consider the basic gadget from Section |III1 with one 
modification that all four servers are now FIFO. Refer to 
Table U for the list of all relevant parameters. As before, we 
will have vertical session(s) going through server Q2- Since 
now horizontal fluid does mix with vertical fluid, we have to 
consider the vertical session(s) explicitly inside the gadget. 

We will allow L vertical sessions coming through server Q2 
(in the full construction, parameter L should define iVvcrt and 
jVrows)- We denote these sessions by Vi, I < i < L. We do not 
want to characterize the arrival pattern of sessions Vi at this 
point. Instead, we shall simply place a bound on the arrivals. In 
particular, we let fi{t) represent the rate at which fluid arrives 
at server Q2 on session Vi at time t, and let f{t) — J2 fii^)- 
We shall assume that 

H2 < Ms + Eg (43) 
f{t) < £l for ah times t (44) 

In general, the vertical sessions are supposed to enter server 
Q2 right after they leave from a similar server from some other 
gadget. For this reason we need to assume that the rates fi may 
depend on 112- To this extent, we will assume the following: 

the intergral / f{t) dt is continuous in /i2 (45) 
for any fixed time T. 

Lemma VI.l Consider the basic gadget under all assump- 
tions in Lemma UII.ll but define all servers to be FIFO. Then: 

(a) There exist parameters {£3, Sg, fi^, K, K') and times 
(TijTsjP) such that given \43M4^ the basic gadget 
fiinctions as shown in Table \n\ 

(b) Part (a) holds for any given K > 20, in which case we 
can choose £s € {5/3K; 20/ K) and set fis = 2es/5. 

(c) Moreover, for any K >10Q and the corresponding choice 
of parameters {£s,6s, K'), and any rates fi(t) satisfying 
SiE5i there exists 1x2 6 [es/60;es/5] satisfying ffS^ 
such that the maximal height of server Q2 is P/[5P]. 

Note that in part (a) one choice of parameters 
(Es, (5s, /i3, A', K') works for any parameter /.t2 satisfying ( l43T l, 
and any arrival rates fi{t) satisfying ( l44l i. The idea is that 
first we choose the above five parameters using part (a), and 
then fine-tune ^2 using (c). Part (b) could be useful to make 
sure that £s is as small as desired. 

Part (c) is important because in the all-FIFO setting the time 
separation between two consecutive rows should be equal to 
the maximal height of server Q2; making this height equal to 
P/h, b <E N makes row i + b synchronized with row i. 

Proof The proof of parts (ab) follows that of 
Lemma UlI. II almost word-by-word, with two modifications: 
• In the second paragraph of Section UlI-CI we need to note 
that the session H traffic mixes with the "vertical" traffic 
in session Q2- However, by (l44l i session H fluid leaves 
server Q2 at rate at least 1 — /i2 — e^. Since this is larger 
than 1 — /i3, we start to get a buildup in server Q3. 
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• For part (a) we need to observe that times Ti, and 
Tp depend only on the parameters {K,£s,5s) and are 
independent of yU2 and the flow rates To see this for 
time Ti, note that for any time t < Ti, height hi{t) is 
given by 

hi{t) = l+ I {di{t) -l)dt, (46) 

where d4{t) is uniquely determined by system jl im4b . 
and therefore by parameters {K,es,Ss)- 

We need more work to prove part (c). Recall that we assume 
K > 100; accordingly, by part (b) we can choose Es < 1/60. 
We'll state explicitely where we need this condition. 

We vary in the interval [sg/SO; es/5], keeping all other 
parameters fixed; note that any such ^2 satisfies ( l43l ) as long 
as £s < 1/5, so all of the previous analysis applies. 

The maximal height of server Q2 becomes a function of 
fi2; we denote this height by i?2(/^2)- Intuitively, this is a 
continuous function. Let us quickly verify that this is indeed 
so. Recall that Ti is fixed. For any time t G [0;Ti], 

h2{t) = / (ti2 + f{t))dt, (47) 

where f{t) = Ef=i Since ^2(^*2) = h2iTi), it is a 

continuous function of /i2 as long as we assume (l45T l. 

We claim that l/ss < Ti < S/ss- Indeed, at any time t G 
[0;Ti], server Qi drains at rate 1 — d4{t), which by ( fT6b lies 
between ^/(l + ^ ~ Ma) and es/(l + — Ms)- Hence the 
time Ti at which server Qi drains satisfies, 

1 1 + £s - M3 ^rr. ^ 1 + f " M3 ^ 3 

~2 

claim proved. 

By ( |47] i. at any time t e [0;ri] height h2{t) increases at 
rate that lies between ^2/(1 — M2) and {fi2 + £s)/(l ~ ^2)- 
The latter number is at most 2/i2 whenever Eg < 1/60 and 
M2 G [£s/60; £5/5], which matches our initial assumptions; 
the proof is a simple but tedious calculation which we omit. 
Therefore 

M2/es < ^^2(^2) < 6m2/£s, 

so, in particular, i72(£s/60) < 1/10, and H2{ejb) > 1/5. 

Finally, let us choose an integer b = [5P]. Since P > 1, 
we have P/b £ [1/10; 1/5]. Since i?2(M2) is continuous, there 
exists a fi2 & [es/60;es/5] such that ^2(^2) = P/b- □ 

Unfortunately, we are unable to construct an example where 
the replenishing servers are strictly FIFO. The main difficulty 
is that for a FIFO replenishing queue, when the vertical 
sessions are creating a buildup of fluid a small amount of 
session H fluid will be served. Since we would like the session 
H fluid to start traversing the row only after all the vertical 
session fluid has left the replenishing queue, this small amount 
of session H fluid is served too early. 

By choosing parameters appropriately we can ensure that 
the amount of session H fluid that is served early is arbitrarily 
small. However, as long as the amount is nonzero it will affect 
the operation of all the subsequent gadgets in the row. We 
conjecture that the effect will be minimal and will not affect 



the final result. We believe that an interesting open problem 
is to prove this conjecture. 

Vll. Conclusions 

In this paper we have shown that natural flow control 
schemes can create oscillations when interacting with net- 
works of queues. These oscillations occur because the queue- 
ing dynamics affect the rate at which data passes through a 
server and so the arrival rate of a session at a server can be 
different than the external session injection rate. In particular, 
our example is always feasible in the sense that the total 
injection rate of all sessions passing through a server is never 
bigger than the service rate of that server This provides a 
contrast with previous work showing oscillations of TCP. 

A number of open questions remain. The first set of 
questions concerns elaborating our construction. As already 
mentioned, we would like to extend our example to the case 
in which all servers are FIFO. Also, we would like to extend 
Theorem 11.21 ('same damping parameter for all sessions') to 
a wider class of flow control functions, most notably to 
multiplicative decrease. Lastly, we would like to accomodate 
TCP Vegas-type schemes (see [23], [3], [7]) where the flow 
rate is increased/decreased at a rate of l/(ri(t))^ depending 
on whether or not there is congestion on the path of flow i. 
Here ^{{t) is the end-to-end-delay on the path of flow i. 

On the other hand, we wonder what would be a good way 
to break our construction. In particular, it would be nice to 
prove a complimentary convergence result for our model under 
minor restrictions on the initial state, the way we fine-tune 
parameters, or the underlying network topology. 

Another area for future study is reducing the complexity of 
our example. We note that much of this complexity arises from 
the fact that we wish to recreate our initial conditions exactly 
so that we can create a infinite sequence of oscillations. We 
are curious if there is a much simpler example in which it 
is possible to create a large finite number of oscillations. In 
particular, we wonder if it might be possible to create such 
examples on more natural topologies. 

Another open question relates to the "stablility" of our 
oscillating example. That is, if we slightly perturb the state 
of the system, will the oscillations persist? We would also 
like to investigate if our example can be applied to networks 
utilizing different scheduling disciplines. For example, we 
wonder whether TCP-like flow control can exhibit oscillations 
when interacting with a network of servers that schedule 
flows according to the Generalized processor sharing scheme 
(see [24], [25], [26]). Another feature of our example is that 
it utilizes large buffers. Whenever a queue builds up the 
participating sessions recognize congestion and then reduce 
their injection rates. It would be interesting to know if our 
example can be adapted to a small buffer scenario in which 
fluid is dropped when congestion occurs. 

Lastly, we would like to investigate to best way to prevent 
oscillations via joint flow control and network scheduling. A 
promising approach in this direction is the Greedy primal-dual 
algoridim of Stolyar [21]. 
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Appendix I 
Continuity issues 

Let us return to Section IIII-CI and consider the integral 
in ( fTSI ) as a function of e that we denote F{e). We show that it 
is continuous in e. We will use the notation from Section Hill 

Claim I.l F{e) is continuous on [ei,e2]- 

We will need some basic facts from two-variable calculus. 

Fact 1.2 For sets S, S' C M^, consider functions f : S ^ R 
and (7 : S" — > R. Let h(x, y) ~ /(x, g{x, y)) and assume that 
this function is well-defined on S' , i.e. that 

{x, g{x, y)) G S for any (x, y) G S' . 

If g is continuous at some point (x^y) G S', and f is 
continuous at the corresponding point {x, g{x,y)) G S, then 
the composition h is continuous at (x, y). 

Fact 1.3 For a closed rectangle S C M^, let x{e, t) be a 
function 5 ^ R which is continuous on S and increasing in t. 
Then writing Xe(t) := x{e,t), the function f{e,x) := xj^(x) 
is continuous on the set {(e, : x ~ a;(e, t) and (e, t) G S}. 

Fact 1.4 Consider a compact set 5 C such that for any 
point {x,y) G S, the segment {x} X [0,y] lies in S. Suppose 
a function / : — > R ;s continuous on S. Then the integral 
g{x, y) := f{x, y) dy is continuous on S. 

Equipped with these facts, let us prove Claim 11.11 Recall 
that e{t), the injection rate into session H at time t, is really a 
function of two variables, time t and parameter e. For clarity, 
let us denote f{e,t) := e{t). This function is defined on 
the closed rectangle R — [ei;e2] x [0;2A']; it is continuous 
by (AE). 

By (11111121) and plugging in /i2 = 2e/5 we have 

Xeit) := t + hi{t)=t{l-2s/5)+ f f(e,t)dt. 

Jo 

This is a function of two variables {e,t); by Fact 11.41 it is 
continuous on R. 
Consider the set 

R* := {{e,x) : X = Xeit) and {e,t) & S} 

= { (e, x) : £ G [£i; £2] and x G [0; Xe{2K)] }. 

Let us treat Xe{t) as a (parameterized) function of one variable, 
t, and consider its inverse xj^{x). This inverse is well-defined 
for all (£, x) G R*. We can view it as a function of these two 
variables; by Fact 11.31 this function is continuous on R*. By 
Fact [O] it follows that g{e,x) := f{e,x^^{x)) is continuous 
on R*, too. 

Let us recall that the rate d4{t) depends on time t and 
parameter e; accordingly, let us write d^it) = di{e,t) Then 
we can re-write (fl4l i as 

^ / A 1 ^ 2£/5 

dAe.x) = — 7 ;— . 

' 5(£,.t) + 1-2£/5 
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By Fact 11.21 this is also a continuous function on R*. By 
Fact 11.41 it follows that the function 

F{e,x) := / d4{e,x)dx 
Jq 

is continuous on R* . Finally, recall that 

F{e) = F{s,T{e)), where T{e) = K/{1 - 2e/5). 

By Fact 11.21 F(e) is continuous on [£1,52], as required. This 
completes the proof of Claim 11.11 
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